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One of the goals of higher education is to develop 



the students' ability to think critically* However, little research 
has been done to indicate the impact of college on students 1 critical 
thinking skills* A study conducted at a midwestern university 
measured the critical thinking capability of college seniors* 
Subjects, 37 volunteers derived from a previous study, had their 
critical thinking ability measured by an original essay question 
designed to force the students to indicate what cognitive processes 
they go through, and what questions they ask when engaged in critical 
evaluation* The analysis of their responses was divided into two 
scoring categories: specific criticisms and general criticisms* The 
results indicated (1) that 40% of the sample identified at least one 
appropriate general criticism and 17% identified more than one; (2) 
the students' specific criticisms showed a general failure to focus 
on definitional issues; (3) that many seniors lack fundamental 
critical thinking skills; and (4) that while most students were able 
to identify some flaws in statistical reasoning, they generally 
failed to recognize ambiguities, questionable assumptions, and value 
preferences — important components of critical evaluation* (MS) 
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One of the goals of higher education is to develop the students' ability to think critically. 
Remarkably, little research has been done to indicate the impact of college on students' 
critical thinking skills. A study (Keeley and Browne, 1986) conducted at a midwestern 
university measured the critical thinking capability of college seniors. While the study 
indicated that most students are able to identify some flaws in statistical reasoning, they 
generally failed to recognize ambiguities, questionable assumptions, and value preferences, 
important components of critical evaluation. 

The sample of thirty-seven volunteers was derived from a previous study (Keeley, Browne, 
and Kreutzer, 1982); they had a mean ACT score of 24.9 and represented the Colleges of 
Education, Arts and Sciences, Business Administration, and Health and Community Service. 

Students 1 critical thinking ability was measured by an original essay question designed to 
force the student to indicate what cognitive processes the student goes through, and what 
questions he asks when he engages in critical evaluation. The rationale for such an approact. 
was inspired by a concern for generalizability. The real world seldom poses multiple choice 
questions to the graduate; rather, he must critically evaluate using his own set of standards. 

During the spring semester, students were presented with a 550 word essay that argued 
college attendance was both a waste of time and money. The essay was selected because it is 
replete with errors in the use of evidence, ambiguities, questionable assumptions, and 
important omitted information. It has also been used in numerous other investigations, and 
its level of difficulty is comparable to other undergraduate materials. The participants were 
instructed to: "Critically evaluate the following article. Evaluation entails judgements 
about the value of material and methods for a given purpose. It involves the use of appraisal. 
In this experiment, your purpose should be to determine how well the author has supported 
his conclusion by applying criteria you feel are appropriate for making such a judgement." 
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The analysis of their responses was divided into two scoring categories-specific criticisms 
such as, H a sample of 300 is too small, 11 and general criticisms, for example, "the author 
persistently overgeneralizes." Responses were analyzed -paragraph by paragraph," and 
-critical question by critical question," to determine whether the students were sensitive to 
the essay's reasoning problems. The following critical questions were used: 

1 . What words or phrases are ambiguous? 

2. What are the value conflicts and assumptions? 

3. What are the definitional and descriptive assumptions? 

4. How good is the evidence: Are the samples representative and measurements valid? 

5. How good is the evidence: Are there flaws in the statistical reasoning? 

The results indicated that 40% of the sample identified at least one appropriate general 
criticism; 17% identified more than one. Most general criticisms fit one of two categories, 
the questioning of the basic value assumption of the purpose of college (to achieve financial 
or academic enrichment), and concerns about omitted information, (e.g., "article is one 
sided," "article lacks enough facts.") Overall, more than half of the students failed to apply 
critical thinking to comment on the article's loose use of the term "college," a definition 
central to the conclusion of the article, or question any other ambiguous terminology. 

The results of the specific criticisms scores indicated a general failure of students to focus 
on definitional issues. Students demonstrated minimal awareness of assumptions not related 
to statistical arguments. Sixty-five percent indirectly recognized assumptions; 65% were 
similarly able to locate at least one flaw in statistical reasoning. Validity of measurement 
was questioned by only 19% of the sample, despite the fact that multiple measures cited in 
the essay were of very questionable validity. This failure to question the validity of 
measurement is especially important because empirical support in behavioral science 
'•larature is often based on questionable measurements. 

It seems that students looking for reasoning errors pay closer attention to numbers than 
words. They do not concentrate on ambiguity and vagueness. They are also unable to 
explicitly pinpoint assumptions. Assumption identification has been stressed in both 
critical thinking tests and texts, but the seniors 1 test results demonstrate a deficiency in 
this important critical thinking skill. While the students were capable of identifying some 
statistical flaws, they missed several others, particularly the problem of sample bias. Only 
25% recognized the author's underlying values and their subsequent impact on the article. 

While this study's sample was limited in size and breadth, the resulis are striking enough to 
serve as a data base for further investigation. The findings clearly indicate that many 
seniors lack fundamental critical thinking skills. To improve the thinking skills of 
students, professors should be aware that the traditional curricula does not guarantee the 
internalization of critical thinking skills. Direct training combined with practice and 
reinforcement is needed to facilitate the development of critical thinking skills. 
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